Learning to Play Soccer using Imitative Reinforcement

The reinforcement framework is a principled approach for agents learning to act in an environment. In the long run, reinforcement learning finds optimal policies. However, a physical agent, such as a humanoid robot, acting in the real world can perform only a limited number of trails, and consequently has only access to limited experience. With such limitations, the exhaustive exploration of high-dimensional state and action spaces is not feasible. One approach to this dilemma is to utilize experiences of other agents by imitating their behavior. If the agents are sufficiently similar, this can speed-up learning dramatically. We propose to give the learning agent access to the Q-values of an experienced agent. The learner combines them with its own Q-values in order to determine its policy. This should head-start learning. We plan to evaluate the effects of this knowledge transfer in a task derived from the RoboCup soccer domain using a humanoid robot.